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SUMMARY 


This  report  describes  the  Shiftwork  Scheduler  Interface  of  the  24/7  Operational  Toolset. 
The  toolset  was  based  upon  the  Sleep.  Activity.  Fatigue,  and  Task  Effectiveness 
(SAFTE,M;  Hursh  et  al.,  2004).  The  SAFTE,M  model  predicts  cognitive  performance 
level  based  upon  sleep,  circadian  rhythm,  and  sleep  inertia.  This  specific  interface  of  the 
toolset  was  designed  to  aid  in  the  construction  of  regular,  rotating,  24/7  shiftw  ork 
schedules.  The  interface  design  approach  was  iterative,  involving  several  meetings 
among  subject  matter  experts  (SMEs).  interface  software  designers  and  evaluators.  The 
first  meeting  was  for  the  purpose  of  requirements  analysis,  in  which  the  designers  elicited 
task  information  from  the  SMEs.  The  second  meeting  included  a  walk-through  of 
storyboarded  and  preliminary  software,  in  which  the  SMEs  provided  feedback  to  the 
designers  and  evaluators.  The  final  meeting  was  for  the  purpose  of  an  "inspection 
evaluation”  of  the  interface  by  the  SMEs  and  evaluators.  This  interface  was  based  upon 
task  analyses  of  AF  Security  Forces  shiftwork  schedulers  who  also  served  as  our  SMEs. 

Our  iterative  sessions  with  SMEs  indicated  that  the  shiftwork  scheduler's  interface 
should  (a)  provide  a  mechanism  that  allows  schedulers  to  make  informed  decisions  with 
respect  to  all  of  the  principles  and  components  of  shiftwork  scheduling,  and  to  make 
“what-if '  comparisons  across  various  alternative  schedules;  (b)  make  comparisons  in 
terms  of  a  specific  schedule's  compliance  with  the  shiftwork  scheduling  principles  and  in 
terms  of  points  within  a  schedule  that  are  particularly  fatiguing,  identified  by  the  SAFTE 
model;  (c)  suggest  countermeasures  to  be  used  when  mishap  risk  is  higher  than  normal; 
(d)  constrain  the  potentially  infinite  possibilities  for  shiftwork  schedules  to  those  that 
make  the  most  sense  in  terms  of  nine  shiftwork  principles;  (e)  be  both  minimal  in  scope 
and  provide  extensive  optional  guidance  with  respect  to  the  nine  principles  and  nine 
components  of  shiftwork  scheduling;  (1)  make  the  shiftwork  scheduling  manual  available 
for  study  and  review:  (g(  identify  clearly  the  “fatigue  points"  in  the  proposed  schedule; 
(h)  allow  side-by-side  comparisons  of  several  candidate  schedules.  The  walk-through 
and  inspection  evaluation  processes  indicated  that  most  of  these  requirements  were  met 
reasonably  well  and  that  potential  users  were  able  to  operate  the  interface  reasonably 
easily. 
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24/7  Operational  Effectiveness  Toolset: 

Shiftwork  Scheduler  Interface 

INTRODUCTION 

This  project  was  aimed  at  designing  several  interfaces  for  a  browser-based  tool  that  could  help  in 
the  construction  of  work-rest  schedules:  the  Fatigue-Performance  Assessment  System  (F-PAS). 
The  tool  was  based  upon  the  Sleep,  Activity,  Fatigue,  and  Task  Effectiveness  (SAFTE™;  llursh 
et  al„  2004)  model.  The  SAFTE,M  model  predicted  cognitive  performance  level  based  upon 
sleep,  circadian  rhythm,  and  sleep  inertia.  Previously,  the  SAFTE™  model  had  been 
implemented  in  the  Windows  software  named  the  Fatigue  Avoidance  Scheduling  Tool™ 
(FAST™;  Eddy  &  llursh.  2001, 2006).  This  report  describes  the  development  of  the  F-PAS 
interface  for  regular  work-rest  schedules  normally  associated  with  shiftwork  and  is  called  the 
Shiftwork  Scheduler  Interface  (SSI).  The  tool  contained  a  set  of  rules  that  determine  the  likely 
sleep  times  of  an  individual  for  starting  work  at  specific,  irregular  times  (Foret  &  Lantin.  1972; 
Pollard.  1996;  Reid  ei  til.,  1997).  Once  the  sleep  times  were  determined,  the  model  predicted 
cognitive  performance  effectiveness  for  the  work  interval  or  work  shift. 

The  overall  design  approach  used  in  this  project  was  iterative,  involving  several  meetings  among 
subject  matter  experts  (SMEs),  interface  software  designers  and  evaluators.  The  first  meeting 
was  for  the  purpose  of  requirements  analysis,  in  which  the  designers  elicited  task  information 
from  the  SMEs.  The  second  meeting  included  a  walk-through  of  storyboarded  and  preliminary 
software,  in  which  the  SMEs  provided  feedback  to  the  designers  and  evaluators.  The  final 
meeting  was  for  the  purpose  of  an  “inspection  evaluation"  of  the  interface  by  the  SMEs  and 
evaluators.  This  interface  was  based  upon  task  analyses  of  AF  Security  Forces  shiftwork 
schedulers. 

Prior  to  conducting  the  requirements  assessment  session  described  here,  w  e  were  aware  of  two 
especially  relevant  USAF  reports.  The  first  described  the  Eagle  Look  concerning  shiftwork- 
induced  fatigue  (AF  Inspection  Agency.  2004).  In  short,  this  all-USAF  review  concluded  that 
the  AF  had  no  method  for  detecting  or  tracking  shiftwork-fatigue  related  mishaps,  inadequate 
training  concerning  shiftwork-induced  fatigue,  and  little  or  no  policy  or  guidance  concerning 
shiftwork  scheduling. 

The  second  report  provided  a  principle-based  approach  to  shiftwork  scheduling  (Miller,  2006). 
The  report  discussed  shiftwork-induced  fatigue,  safety,  calendar  arithmetic,  circadian  stability, 
the  Principles  of  Chronohygiene,  shiftworker  satisfaction,  the  number  of  shiftworkers  needed, 
the  basic  structure  of  shiftwork  systems,  and  plans  (rotas),  scheduling  methods  and  examples, 
and  the  effects  of  schedule  changes.  It  cited  the  use  by  USAF  Security  Forces  of  one  specific, 
slow  rotation  plan.  That  plan,  or  rota,  was  called  the  "Panama"  or  “every-other- weekend-off' 
rotation.  It  is  described  under  Requirements  Analysis,  Security  Forces  Practices  in  the 
Discussion  section. 
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METHODS 


A  more  detailed  explanation  of  the  methods  used  here  may  be  viewed  in  the  companion  technical 
report,  24/7  Operational  Effectiveness  Toolset:  Usability  Assurance  Plan  (Miller.  Eddy  &  Moise, 
200S(.  The  SME  sessions  were  conducted  by  NT1  under  the  auspices  of  the  AF  Security  Forces 
Center.  Lackland  AFB.  Texas,  and  were  held  in  their  offices  and  laboratory  at  Lackland  AFB. 


REQUIREMENTS  ANALYSIS 

The  first  session  was  held  at  the  Security  Forces  (SF)  headquarters  building  at  Lackland  AFB  on 
24  and  25  July  2006.  Three  SF  SMEs  participated:  two  Senior  Master  Sergeants  and  a  Master 
Sergeant. 

The  agenda  for  the  first  session  was  as  follows: 

•  Introductions,  goals  and  objectives  (NT1| 

•  Fatigue  Management  overview  (NT1) 

•  Participant  overview  of  their  tasks  and  background  (SMEs) 

•  Introduction  to  task  analysis  (TA)  and  procedures  (NTI) 

•  TA:  goals  and  processes 

•  TA:  tasks 

•  Review  of  goals  and  tasks 

•  Usability  assessment  of  FAST™ 

To  initiate  discussions.  NTI  described  fatigue  terminology  and  fatigue  management  concepts  to 
provide  a  common  understanding  of  fatigue  and  performance.  The  researchers  used  FAST™ 
graphs  to  briefly  discuss  and  demonstrate  the  impact  of  fatigue  on  cognitive  performance. 


WALK-THROUGH 

From  the  user  task  descriptions  and  the  requirement  assessment  (RA).  we  created  scenarios  for 
user  testing  and  review.  In  the  walks  through  the  draft  interface,  the  designers,  potential  end 
users,  and  evaluators  worked  together  to  step  through  typical  tasks  for  which  the  interface  was 
designed.  The  users  were  given  preliminary  training  in  fatigue  management,  on  the  capabilities 
of  the  browser-based  tool  called  Fatigue-Performance  Assessment  System  (F-PAS)  and  on  the 
draft  SSI.  The  designers  demonstrated  the  tool  by  entering  data  for  a  fictitious  scenario. 
Questions  and  discussions  accompanied  each  screen  of  the  tool. 

We  roughly  estimated  the  total  number  of  usability  problems  in  the  interface  from  the  number  of 
problems  (E)  identified  during  the  walk-through.  Assuming  a  detection  rate  of  about  30%  in  the 
walk-through,  the  total  number  of  problems  would  be  about  equal  to  E  divided  by  0.30  (Bailey, 
1997). 

INSPECTION  EVALUATION 

In  the  inspection  evaluation,  representative  end  users  tried  to  do  typical  tasks  with  the  interface 
while  a  designer/evaluator  watched,  listened,  and  took  notes.  We  wished  to  identify  usability 
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problems,  collect  quantitative  data  on  participants'  performance  and  determine  the  participants' 
satisfaction  with  the  product.  More  specifically,  we  wished  to  leam: 

•  Could  the  test  participants  complete  the  relevant  tasks  successfully? 

•  How  long  did  it  take  the  participants  to  do  each  subtask? 

•  Did  the  participants  perform  well  enough  to  meet  their  usability  objectives? 

•  1  low  satisfied  were  the  participants  with  the  interface? 

•  What  changes  were  needed  to  make  sure  that  the  interface  would  enable  more  users  to 
perform  more  successfully? 

These  latter  questions  were  addressed  through  the  questionnaire  shown  in  Appendix  A.  The 
observer  collected  detailed  data  using  the  scoring  sheet  shown  in  Appendix  B.  The  user  was 
provided  with  a  separate  document  showing  the  34  questions  in  the  scoring  sheet. 
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RESULTS 


REQUIREMENTS  ANALYSIS 

The  SMEs  provided  a  brief  overview  of  their  role  in  shift  work  scheduling,  and  outlined  who 
might  be  the  primary  users  of  our  new  fatigue  management  tool.  They  indicated  that  their 
security  forces  had  three  security  missions: 

•  Nuclear  assets,  the  highest  priority 

•  Garrison,  working  at  well-established  USAF  sites 

•  Deployed  forces,  working  in  combat  areas 


Potential  Users 

Regarding  potential  users,  the  SMEs  indicated  that  the  Security  Operations  Officer  (0-3  to  0-4) 
and  the  Operations  Superintendent  (E7  to  E-8)  would  be  primary  and  the  software  interface 
would  need  to  meet  their  needs.  Additionally,  they  indicated  that  the  interface  design  should 
also  consider  the  needs  of  the  Flight  Sergeant  and  Commander  since  they  would  likely  use  the 
tool  as  well.  They  indicated  that  it  would  not  be  necessary  to  consider  the  needs  of  security 
personnel  at  the  grade  of  E-4  and  below  in  the  design  process. 

In  discussing  the  skill  and  abilities  of  our  users,  the  SMEs  felt  that  users  would  be  E-5s  or  higher. 
They  would  be  high  school  graduates  with  some  college  work,  familiar  with  the  internet,  and 
familiar  with  browser-based  tools.  An  E-6  w  ould  have  had  5-10  years  of  security  experience 
before  becoming  an  Operations  Superintendent.  Security  personnel  receive  training  at  AF 
Regional  Training  Centers  and  at  each  major  command. 

The  Operations  Officer  and  Operations  Superintendent  set  the  general  orientation  for  post 
assignments  after  a  site  is  considered  secure.  They  also  set  the  shift  start  times.  The  Flight 
Sergeant  decides  who  works  when  and  where.  The  SMEs  indicated  that  schedules  were  created 
out  of  the  post  requirements  and  work  force  available.  Other  inputs  were  secondary  .  Generally, 
schedules  were  created  w  ithout  considering  whether  off-duty  days  occurred  on  weekends. 


Task  Analvsis 

Next,  the  task  analysis  (TA)  procedures  were  introduced  and  the  process  was  begun.  We  asked 
the  SMEs  to  list  their  goals  and  processes  along  with  the  tasks  that  accomplished  them.  Each  of 
eight  goals  and  processes  was  discussed  thoroughly.  An  overview  of  the  task  analysis  results  is 
shown  in  Table  I.  Discussion  with  the  SMEs  focused  on  how  schedules  were  created:  thus,  the 
automation  potential  for  tasks  was  not  considered  in  much  depth  during  these  discussions. 
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TABLE  I.  Overview  of  the  task  analysis  results 


■ 

PROCESSES/GOALS  STEPS 

TASKS  TO  ACCOMPLISH 

SUGGESTED 

AUTOMATION 

Site  survey 

A.  Terrain  analysis:  Determine 
security  boundary,  threats. 

TAOR* 

B.  Determine  duration 

C.  Determine  weapons  available  for 
defense 

D.  Determine  security  requirements 
to  calculate  required  manpower 

Prioritize  post  assignments 

L.  Determine  work  demand  per 
security  post 

F.  Develop  post  priority  chart 

Determine  personnel  needed 

G.  Determine  skill  levels  required 
per  position 

H.  Consider  manning  ratio  (post 
manning  factor) 

I.  Determine  rotation  schedule 
(Consider  rota  or  plan) 

J.  Determine  whether  to  use  S-  or 
12-hour  shift  length 

K.  Determine  shift  change  time 

Provide  scheduling 
options  and  schedule 
comparisons 

Determine  number  of  elements 

L.  Consider  work' rest  schedule 
(days  oft)  &  mandated 
components 

Provide  scheduling 
options  and  schedule 
comparisons 

5. 

Leadership  validation  of 
proposal 

M.  Incremental  changes 

Provide  schedule 
editing  functions 

1 

Implementation 

N.  Dissemination  of  duty  schedules 

O.  Consider  day  of  week  for  start 

Schedule  feedback  (expected 
evolution) 

P.  Observe  performance  and  adjust 
based  on  experience 

Provide  schedule 
editing  functions 

Maintenance 

Q.  Attend  to  changing  schedule 
requirements 

Analyze  impact  of 
change  on  replacement 
person 

Notes:  This  Fable  starts  with  the  firsts  steps  in  shift  work  scheduling  and  progresses  through  to 
approval,  schedule  feedback  and  schedule  maintenance.  Data  in  each  column  move  from  left  to  right, 
from  global  Process'' Goa  Is,  to  specific  Tasks  that  are  directly  related  to  those  stated  processes- goals. 

The  last  column  addresses  how  an  automated  tool,  such  as  FAST1  Vl  might  facilitate  the  tasks. 

*TAOR:  tactical  area  of  responsibility;  the  area  that  the  defense  force  commander  can  control  through 
organic  hcavy/light  weapons  fire. 

Generally,  the  scheduling  process  emulated  the  following  logic: 

1 .  Survey  the  site  to  be  protected,  determining  the  work  demand  and  the  human  and 
weapons  resources  needed 

2.  Prioritize  all  of  the  security  posts  that  would  be  manned 
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3.  Calculate  the  specific  numbers  and  kinds  of  SF  personnel  needed  to  do  the  job  and  their 
shirt  schedule(s) 

4.  Determine  how  to  break  the  needed  personnel  into  rtights  and  elements  (shiftwork  crews) 

5.  Secure  minor  revisions  and  approval  from  senior  leadership 

6.  Implement  the  final  plan 

7.  Monitor  the  adequacy  of  the  schedule 

S.  Make  changes  to  the  schedule,  as  needed 

Concerning  the  eighth  step,  ail  hoc  modifications  to  shiftwork  schedules  occurred  in  response  to 
varying  security  demands.  The  modifications  were  implemented  by  increasing  the  ratio  of  work 
days  to  days  off  and  by  ceasing  to  guard  low  priority  posts  temporarily  in  favor  of  higher- 
priority,  transient  demands  for  security.  Generally,  transiently-increased  security  demands  could 
not  be  met  by  increased  numbers  of  Security  Forces  personnel  due  to  their  scarcity. 

On  the  second  day  of  the  session,  we  began  by  reviewing  the  eight  goals  and  processes  and  their 
associated  tasks  to  make  corrections.  Table  1  reflects  the  few  changes  made  during  the  review. 
Next,  the  SMRs  provided  additional  information  related  to  each  of  the  tasks:  what  or  who  started 
or  initiated  each  task  or  group  of  tasks,  what  media  was  used  to  complete  the  task,  the  input 
information  required  of  the  task,  and  what  the  output  product  was.  This  information  is  captured 
in  Table  II. 

With  respect  to  shiftwork  scheduling.  Tasks  A  through  G  in  Table  II  deal  with  personnel 
selection  issues  that  precede  actual  scheduling.  Tasks  II  through  J  deal  with  an  essential 
question:  Are  there  enough  people  (of  the  needed  skill  levels)  to  support  24, 1 7  operations?  Tasks 
K  through  O  address  actual  shiftwork  scheduling.  Tasks  P  through  V  deal  with  the 
implementation  and  maintenance  of  the  schedule. 


TABLE  II.  Task  characteristics  within  each  goal 


TASKS  WITHIN 
GOALS/PROCESSES 

DIRECTIVES 

MATERIALS 

AND  MEDIA 

INPUTS 

OUTPUTS 

1.  Site  survey  tasks 

Air  Tasking 
Order.  Air  Staff, 
Major  Command 

Electronic 

Tasking  order 
includes 
images  &  text: 
the  "Big 
Picture" 

A.  Terrain  analysis: 
establish  security 
boundary,  define 
threats.  TAOR 

B.  Determine  duration 

C.  Determine  weapons 
needed 

D.  Determine  manpower 
requirement 

2.  Prioritize  post 
assignments  (tasks) 

Own  unit 
analysis  of  site 

6 

Approved  fur  public  release:  di'tnbulwn  unlimited.  Public  Allans  Case  Ilk  mi.  09-295. 29  lune  21X19 


TASKS  WITHIN 
GOALS/PROCESSES 


E.  Determine  work 
demand  per  post 


F.  Develop  post  priority 
chart 


3.  Determine  personnel 
needed  for  posts 


C».  Determine  skill  levels 
required  by  position 


H.  Consider  manning  ratio 


I.  Establish  rotation 
schedule 


J.  Choose  8-  or  12-hour 
shift  length 


K.  Consider  change  time 


4.  Determine  number  of 
elements 


L.  Consider  work' rest 
schedule  (days  oft) 
mandated  components 


5.  Leadership  validation  of 
proposal  tasks  (at  squadron 
level) 


M.  Incremental  changes 


6.  Implementation  tasks 


N.  Dissemination  ot  duty 
schedules 


O.  Consider  day  of  week 
for  start 


7P.  Schedule  feedback 
(expected  evolution)  tasks 


intenancc  tas 


DIRECTIVES 


AH  31-101, 
DODC-5210 
.4 1  -M 


Operations 

officer 


MATERIALS 
AND  MEDIA 


Elcctronic’papcr 


Elcctronic/papcr 


Final  electronic 


Paper*  pencil, 
white  board 


INPUTS 


Unit  Manning 
Document 


Leader's 

planning 


OUTPUTS 


No.  of  people 
&  skills 


Post  Priority 
Chart 


Allocation  of 
flights 


Duty  roster 
template 


12 -hour  is 
default 


Traffic.  desert 
temps.,  meals 


#  of  personnel  & 
post  manning 
requirements: 


Time:  usually 
same  for  all 
elements 


on:off  dutv 

* 

ratio  =  3:1, 
4:1,  5:1 


Paper’ pencil, 
whiteboard 


Final  electronic 


Paper*  pencil/ 
whiteboard 


Verbal  textual 
input  (duty 
rosters) 


Y  exchange 


Duty  roster 
template 


Duty  roster 


aper,  electronic 


Ops  Officer 


Input  from  those 
scheduled 


Paper*’ pencil, 
whiteboard,  final 
electronic 


Duty  roster  Revised  duty 


tv  roster 


Other  information  offered  during  the  sessions  included  the  following: 

•  USAF  Security  Forces  were  suffering  from  a  large  work  force  shortage. 

•  Work-rest  schedules  usually  lasted  about  30  day's.  Generally,  the  Security  Forces 
embrace  this  slow  rotation  philosophy. 
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•  Morale  was  a  big  issue  for  the  security  forces  leading  to  retention  resistance.  They 
attributed  low  morale  to  regularly  losing  one  day  of  leave  each  month  due  to  extra  duties. 

•  They  conducted  training  on  “off'  days  usually  one  day  per  month.  (The  concept  of  a  5Ul 
element  was  discussed  for  accomplishing  training.) 

•  The  number  of  people  and  the  number  of  posts  determined  the  duty  roster  template. 

•  The  ratio  of  on-  versus  off-duty  personnel  was  generally  3: 1 . 

•  Raven  forces  sometimes  used  No-go  drugs  (approved  sleeping  pills),  under  medical 
supervision,  for  rest  during  difficult  missions  or  poor  sleeping  conditions.  ‘‘Air  Mobility 
Command's  Phoenix  Raven  program,  implemented  in  1997.  consists  of  teams  of 
specially  trained  security  forces  personnel  dedicated  to  providing  security  for  AMC 
aircraft  that  transit  high  terrorist  and  criminal  threat  areas."  (usmilitary.about.com) 

•  Security  Forces  personnel  venerated  the  Raven  program  for  their  fast  deployments. 

Raven  teams  deployed  approximately  once  every  two  to  three  months. 

Documents  referenced  during  the  Task  Analysis  included: 

•  DoD  C-52 1 0.4 1  -M.  Nuclear  Weapon  Security  Manual  (U) 

•  AF  3 1  -101 .  The  Air  Force  Installation  Security  Program,  provides  information  on 
required  staffing  for  various  posts  (security  sites).  The  Air  Combat  Command 
supplement  contains  additional  useful  information  on  alternate  posts. 

•  AF  31-302.  Air  Defense  1  landbook.  provides  guidance  for  the  collective  skills  required  to 
carry  out  the  Air  Force  air  base  defense  mission. 

•  AF  31-305.  Security  Forces  Deployment  Planning  Handbook,  provides  Security  Forces 
Commanders  and  planners  with  the  basic  requirements  for  deployment  planning  and 
execution  in  support  of  operational  and  contingency  plans.  Aerospace  Expeditionary 
Force,  and  contingency  deployment  and  redeployment  operations.  It  provides  basic 
information  on  Unit  Type  Code  development,  use  and  capabilities. 


Usability  of  FAST1'1 

The  SME's  reactions  to  version  1.5  of  FAST™  were  typical  for  non-scientists.  They  found  the 
graphical  input/output  interface  to  be  confusing  and  the  grid  input  interface  to  be  potentially 
useful.  They  desired  the  ability  to  make  “what-if  comparisons  across  various  schedules  in  the 
output  component  of  the  interface. 


INTERFACE  PLANNING 

Previously,  we  had  considered  several  aspects  of  shiftwork  scheduler  output  interfaces  and  their 
advantages  and  disadvantages  for  application  in  a  24/7  Scheduler  (Eddy  and  1  lursh.  2006). 

These  ideas  for  a  24/7  output  interface  were  the  result  of  the  shortcomings  of  the  FAST,M  output 
display  that  showed  a  schedule  for  only  one  person  or  one  crew.  Shift  work  schedulers  need  to 
see  the  impact  of  their  schedule  on  all  shifts  or  crews  to  compare  the  benefits  and  costs  of 
different  schedules.  The  conceptual  displays  presented  by  Eddy  and  1  lursh  were  intended  to 
allow  schedule  comparisons  across  all  crews.  Unfortunately,  we  were  unable  to  identify  an 

8 


Approved  for  public  release:  disInbulKiu  unlimited.  Public  Allans  Case  file  iki.  09-295. 29  lune  21X19 


algorithm  that  would  calculate  crew  lag  reliably  for  all  possible  shiftwork  schedules.  Thus,  the 
lag  calculation  task  was  not  automated.  Instead,  the  interface  was  designed  to  advise  the  user: 

The  solution  created  from  your  inputs  here  refers  only  to  one  crew  in  a  4-crew  plan.  To 
integrate  all  4  crews  into  a  single  schedule,  find  the  lag  time  manually  from  crew  start  to 
crew  start.  To  do  this,  lay  out  four  copies  of  the  whole  single-crew  cycle  and  slide  them 
against  each  other  until,  scanning  across  the  4  crews,  you  see  that  each  shift  is  covered 
once  per  day.  and  no  more  than  once  per  day.  I  lere  is  an  example.  Note  that  for  4-crew 
solutions,  one-quarter  of  the  days  in  a  cycle  must  be  off(O)  days  for  8-hour  shifts,  and 
one-half  of  the  days  in  a  cycle  must  be  ()  days  for  12-h  shifts. 

The  example  provided  was: 

Because  the  solutions  offered  here  deal  only  with  a  single  crew,  you  need  to  find  the 
number  of  lag  days  before  the  next  crew  starts  the  same  schedule.  Unfortunately,  there  is 
no  magic  formula  (yet)  for  calculating  the  number  of  lag  days.  The  number  of  lag  days  is 
usually  the  value  of  F  (free  days)  from  your  shift  system,  or  one-half  of  F.  It  may  also  be 
all  or  half  the  total  of  W  and  F. 

Draw  a  table  with  one  column  for  each  crew  and  a  line  for  each  day  in  the  cycle.  Write 
in  the  Crew  A  schedule  in  column  1.  from  top  to  bottom.  Count  down  the  expected 
number  of  lag  days  from  the  start  of  Crew  A's  plan.  and.  starting  at  that  point,  pencil  in 
the  same  plan  for  Crew  B  in  the  Crew  B  column,  to  the  right  of  the  Crew  A  column. 
Pencil  in  the  same  plan  for  Crews  C.  D.  etc.,  using  the  same  number  of  lag  days  from  the 
crew  to  the  left,  and  using  the  same  plan.  From  the  bottom  of  each  column,  wrap  around 
to  the  top  of  that  column  and  continue  penciling  in  the  schedule. 

Then,  scan  across  each  day.  If  you  are  using  12-h  shifts,  look  for  one  and  only  one  D  in 
each  day's  line,  one  and  only  one  N  in  each  day's  line.  If  the  column  meets  these  criteria, 
place  a  check  mark  for  that  day  in  a  summary  column  at  the  right  for  that  day.  If  you  are 
using  8-h  shifts,  look  for  one  and  only  one  D.  one  and  only  one  S,  and  one  and  only  one 
N  in  each  line.  If  the  column  meets  these  criteria,  place  a  check  mark  for  that  day  in  the 
summary  column  at  the  right  for  that  day. 

If  all  days  meet  the  criterion,  then  one  crew  fills  each  shift  on  every  day  of  the  cycle. 

This  is  what  you  want.  If  some  lines  do  not  meet  this  criterion  (some  check  marks  are 
missing),  then  go  back  and  try  another  number  of  lag  days.  Make  sure  that  you  are  using 
the  same  plan  in  every  crew  column.  If.  after  trying  all  lag  lengths  from  one  up  to  the 
total  of  the  W  and  F  in  your  system,  you  cannot  fill  each  shift  with  one  crew  each  day. 
you  need  to  re-examine  your  decisions  about  numbers  of  crews  and  numbers  of  shifts  per 
day. 

A  non-table  way  to  find  the  number  of  lag  days  is  to  type  out  the  plan  several  times,  print 
it  out.  cut  it  into  strips,  and  move  the  strips  back  and  forth  next  to  each  other  until  you 
have  only  one  crew  per  shift  per  day.  Your  typing  might  look  like  this  for  four  cycles  of 
the  4-crew.  12-h.  DDNNROCX)  plan: 
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Crew  A:DDNNROOODDNNROOODDNNROOODDNNROO() 

Crew  B.  DDNNROOODDNNROOODDNNROOODDNNROOO 
Crew  CDDNNROOODDNNROOODDNNROOODDNNROOO 
Crew  D:DDNNROOODDNNROOODDNNRO()ODDNNROOO 

Though  we  were  unable  (o  implement  one  of  (he  multi-crew-displays  suggested  by  Eddy  and 
Hursh  (2006).  the  discussion  illustrated  to  us  the  need  for  innovation  in  how  we  would  present 
SAFTE  output  data  to  the  shiftwork  scheduler. 


WALK-THROUGH 

Three  walks  through  the  draft  software  were  conducted  at  Lackland  AFB  with  SMEs  provided 
by  the  SF  Center. 

First  Walk-Through 

Dr.  Miller  walked  several  SMEs  through  the  first  draft  of  the  interface  on  1 1  September  2007. 
The  draft  was  designed  to  place  a  constrained  amount  of  information  about  shiftwork  scheduling 
on  a  single  web  page.  The  material  was  drawn  from  Miller  (2006).  The  main  thrusts  included 
hyperlinked  pop-up  windows  that  presented  somewhat-detailed  information  about  scheduling,  a 
constrained  set  of  know  n  shiftw  ork  plans  and  a  method  for  acquiring  a  simple  analysis  of  a  new 
plan. 

The  draft  was  generally  well  received.  Twenty  one  questions  and  recommendations  resulted 
from  the  walk-through,  suggesting  that  there  might  have  been  as  many  as  70  possible 
improvements  that  could  have  been  identified  (21/.3  70;  Bailey,  1997). 

•  Explain  shift  overlap  clearly. 

•  What  is  the  reference  for  a  relative  risk  of  1 .0  on  the  shift  length  graph? 

•  The  color  red  may  indicate  to  some  commanders  that  the  fatigue  risk  is  too  great  to 
accept.  On  the  other  hand,  some  commanders  may  note  that  they  have  been  operating 
extensively  in  the  "red"  without  mishaps  and  so  may  ignore  it.  Possible  solution  is  to  use 
a  different  color  for  the  known,  pre-dawn  trough. 

•  Support  the  problem  of  under- manning  better. 

•  Emphasize  BAC  equivalency. 

•  Use  a  translation  (truth)  table  for  effectiveness,  BAC  and  relative  risk.  Also,  put  truth 
tables  in  examples  of  shiftwork  schedules. 

•  Add  a  fatigue  countermeasures  (risk  mitigations)  list. 

•  Recommend  not  covering  lower-priority  posts  to  reduce  fatigue  risk. 

•  Use  many  fewer  words.  Look  at  Army  websites  for  ideas  on  game-like  graphics.  Never 
require  the  user  to  scroll  the  page. 

•  Suggest  the  home  page  contain  the  three  satisfaction  principles,  then  three  buttons  for 
selecting  introductory  material,  worked  examples,  or  schedule  creation. 

•  When  displaying  various  shift  choices  highlight  the  four-crew  solution  and  emphasize 
satisfaction  principles. 

•  Add  a  pop-up  for  driver  fatigue  case  law  related  to  off-base  risk. 

•  Use  an  easily  accessible  effectiveness  graph,  with  tool  tips. 
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•  Use  a  threshold  warning:  i.e.,  use  the  dotted  line  display  concept  used  in  FAST. 

•  "Stand-to”  time  occurs  pre-dawn,  when  it  is  known  that  attentiveness  is  poor  and  threat  is 
high.  How  might  we  use  this  info? 

•  Enter  data  using  a  calendar,  as  in  Outlook.  Show  shift  overlap  in  shades  of  gray. 

•  The  crew  number  graph  is  good.  Increase  its  importance. 

•  Consider  Palm  Pilot  (PDA)  use  at  some  point. 

•  The  site  needs  a  guide  to  analyzing  a  present  shiftwork  schedule. 

•  Use  shading  in  shift  plan  tables  to  show  each  shift.  For  example: 


Monday  |  Tuesday 

Wednesday 

Thursday 

Friday 

Saturday 

Sunday 

Day 

— 

Swing 

1*3  1  »2  1 

Note:  All  shifts  are  8  hours  long  unless  noted 


In  response  to  these  questions  and  recommendations,  the  following  changes  were  made  to  the 
interface: 

•  The  page  was  split  into  five  smaller  pages:  Home.  New  SF  Plan  [Security  Forces], 
Known  Plans.  New  Plan,  and  Additional  Information.  The  I  lome  page  emphasized 
having  enough  people,  using  four  crews  and  paying  attention  to  worker  satisfaction. 

•  A  pop-up  was  added  that  explained  shift  overlap,  and  it  was  linked  from  both  the  Home 
and  Additional  Information  pages. 

•  The  color  blue  was  used  for  the  known,  pre-dawn  trough,  instead  of  red  on  the  output 
page. 

•  The  problem  of  under-manning  was  emphasized  more  by  adding  both  a  pop-up  and  a 
calculator.  These  were  linked  from  both  the  Home  and  Additional  Information  pages. 

•  A  fatigue  countermeasures  (risk  mitigations)  list  was  added  in  a  pop-up  and  it  was  linked 
from  the  Additional  Information  page  and  the  output  page.  It  recommended  not  covering 
lower-priority  tasks  to  reduce  fatigue  risk  at  the  remaining  posts. 

•  The  output  page  was  changed  to  use  color-coded  shading  in  a  shift  plan  table  to  show- 
each  shift. 


Second  Walk-Through 

Dr.  Miller  walked  several  SMEs  through  the  revised  pages  on  24  September  2007.  The  new 
design  was  also  well  received.  The  14  new  recommendations  of  the  SMEs  were  to: 

On  the  I  lome  page: 

•  Create  links  to  the  shiftwork  manual  and  the  shiftwork  survey  report. 

•  Add  a  note  about  task-specific  fatigue  and  the  20-minute  vigilance  decrement. 

On  the  New  SF  Plan  page: 

•  Use  the  employment  ratio  calculator  instead  of  a  ratio  assumption. 

•  Add  a  work  force  disclaimer  re  AFPC  manning  policy. 

•  Add  a  line  for  number  of  deployment  days  on  the  ratio  calculator  page. 

•  Countermeasures  pop-up: 

o  Add  more  information  on  the  best  times  to  nap. 
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o  Add  more  information  about  caffeine  use. 

o  Add  a  disclaimer  about  the  Personnel  Reliability  Program  for  the  modafinil 
paragraph. 

•  Analysis  page 

o  Add  a  disclaimer  about  the  average  sleeper  vs.  long  and  short  sleepers, 
o  Add  a  disclaimer  about  the  assumption  that  appropriate  amounts  of  sleep  are  acquired 
during  time  off. 

o  Orient  the  week’s  table  in  the  output  more  horizontally  and  add  space  between  rows, 
o  Add  a  link  to  the  countermeasures  pop-up  after  the  definition  of  "yellow." 

•  Add  FAST  graphs  in  all  known-plan  pop-ups. 

•  Build  a  stand-alone  version  of  the  interface  (no  connection  to  SAFTE  engine  on  server). 

•  In  the  SAFTE  engine,  add  a  shift  overlap  period  that  is  additive  with  the  commute  period. 

In  response  to  these  questions  and  recommendations,  the  following  changes  were  made  to  the 
interface: 

On  the  1  lome  page: 

•  Added  links  to  the  fatigue  countermeasures  and  SAFTE  pop-ups. 

•  Added  links  to  the  shiftwork  manual  and  the  shiftwork  survey  report. 

On  the  New  SF  Plan  page: 

•  Used  the  employment  ratio  calculator  instead  of  a  ratio  assumption. 

•  Added  a  work  force  disclaimer  re  AFPC  manning  policy. 

On  the  Analysis  page: 

•  Added  sentences  about  the  average  sleeper  vs.  long  and  short  sleepers. 

•  Added  a  sentence  about  the  assumption  that  appropriate  amounts  of  sleep  are  acquired 
during  time  off. 

•  Oriented  the  week's  table  in  the  output  more  horizontally  and  added  space  between  rows. 

•  Added  a  link  to  the  countermeasures  pop-up  after  the  definition  of  “yellow." 

On  the  countermeasures  pop-up  page: 

•  Added  sentences  about  sleep  aids,  with  a  Personnel  Reliability  Program  (PRP) 
disclaimer. 

•  Added  a  disclaimer  about  the  PRP  for  the  modafinil  paragraph. 

•  Added  a  few  more  words  about  caffeine  use. 

•  Added  sentences  on  the  best  times  to  nap. 

•  Added  a  sentence  about  the  pre-dawn  “stand-to”  time  in  the  Double  Checking  paragraph. 
Other 

•  On  the  ratio  calculator  page,  added  a  line  for  number  of  deployment  days. 

•  On  the  shift  length  pop-up  page,  added  a  note  identifying  the  reference  for  a  relative  risk 
of  1 .0.  Added  similar  notes  for  other  relative  risk  graphs  in  other  pop-ups. 

•  On  the  SAFTE  pop-up  page,  added  paragraphs  about  task-specific  fatigue  and  the 
vigilance  decrement. 
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•  Created  a  translation  (truth)  table  for  effectiveness,  BAC  and  relative  risk;  linked  from 
Additional  Information  page.  Included  technical  references  for  data  sources. 

•  Added  a  paragraph  concerning  shiftworker  driver  fatigue  on  the  Additional  Info  page, 
with  links  to  local  NUTS  A  materials. 

The  following  tasks  were  postponed: 

•  Build  a  stand-alone  version  of  the  interface  (no  connection  to  SAFTE  engine  on  server). 
This  version  would  lack  the  “New  Plan"  page  and  would  be  hosted  on  a  SF  serv  er. 

•  Use  an  easily  accessible  effectiveness  graph,  with  tool  tips. 

•  Enter  data  using  a  calendar,  as  in  Outlook.  Show  shift  overlap  in  shades  of  gray. 

•  In  the  SAFTE  engine,  add  a  shift  overlap  period  that  is  additive  with  the  commute  period. 

•  Add  a  brief  help  file,  context  sensitive,  explaining  the  rationale  for  each  page. 

•  Change  ratio  calculator  lines  from  fixed  words  to  defaulted  text  boxes. 

•  Protect  from  employment  ratio  <  I  errors. 


Final  Walk-Through 

Dr.  Miller  walked  several  SMEs  through  the  revised  pages  on  19  February  2008.  The  objective 
of  the  meeting  was  to  update  the  SF  Center  personnel  on  the  new  web  site,  show  them  the 
Shiftwork  Scheduler  Interface  on  line  for  the  first  time,  review  changes  made  in  response  to  the 
second  walk  through,  and  plan  for  usability  testing.  These  objectives  were  met. 

The  site  and  the  interface  pages  were  accessible  from  one  of  the  SF  computers  in  the  building. 
For  usability  testing.  The  SFC  agreed  to  recruit  up  to  6  "operations  types"  who  were  experienced 
in  unit-level  (squadron)  schedule  planning.  Three  new.  interesting  issues  came  up  in  discussions. 
First,  the  SMEs  noted  that  the  interface  will  be  good  for  unit-level,  “macro"  scheduling.  Then, 
schedulers  at  the  (light  and  element  levels  may  use  the  Mission  Scheduler  interface  to  decide 
how  to  schedule  personnel  within  12-h  shifts. 

Second,  when  asked  about  alternative  schedules  for  use  in  usability  testing,  the  SMEs  mentioned 
that  a  two-flight,  three-element  plan  with  5  days  on  and  2  days  off  and  12-h  shifts  was  used  quite 
a  bit.  This  may  really  be  a  6-crew  solution  and  SF  may  be  able  to  use  fewer  people  with  a  5- 
crew  system.  This  possibility  should  be  described  in  the  Help  file.  Finally,  the  SMEs  noted  that 
schedulers  simply  do  not  grasp  the  concept  that  changing  from  8-h  shifts  to  12-h  shifts  does  not 
decrease  the  number  of  people  needed.  This  point  must  be  emphasized  in  the  Help  file. 


INSPECTION  EVALUATION 

Six  senior  non-commissioned  officers  participated  as  representative  users:  one  retired  Senior 
Master  Sergeant  (SMSgt;  E-S),  four  SMSgts  on  active  duty,  and  one  Master  Sergeant  (E-7)  on 
active  duty.  All  were  assigned  to  the  SFC,  with  the  retired  SMSgt  working  there  as  a  contractor. 
They  were  tested  individually  on  the  mornings  of  25.  28  and  29  September  2008  at  the  Modeling 
and  Simulation  Branch  of  the  SFC.  A  Lackland  AFB  desktop  computer  with  Internet  access 
(Windows  XP)  was  used  for  all  testing.  There  were  no  login  or  display  format  problems. 
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The  participants'  mean  total  active  duty  time  was  22.8  years  (range,  19  to  25  years).  Their  mean 
“years  spent  in  shiftwork  and/or  night  work  as  worker  and/or  scheduler”  was  15.0  years  (range,  6 
to  20  years).  Only  one  of  the  six  participants,  the  retired  SMSgt,  had  used  FAST  previously,  and 
that  was  for  staff  w  ork  at  the  SFC. 


Interface  Ratings 

The  ratings  (Appendix  A)  provided  by  the  participants  concerning  the  interface  were: 


Dimension 

Median 

Mode 

Range 

Ease  of  application 

2 

2 

1  to  3 

Performance 

1 

1 

1  to  4* 

Overall  function 

1.5 

1 

1  to  3 

•The  single  rating  of  4  reflected  the  single  instance  of 
seivet  accessibility  failuie  desciibed  below 


Where  the  ratings  were  defined  as: 

1 .  Very  acceptable 

2.  Acceptable 

3.  Borderline 

4.  Unacceptable 

5.  Very  unacceptable 

Since  there  was  no  help  system  operating  at  the  time  of  testing,  the  “support”  dimension  was  not 
rated. 


Times 


The  mean  times  spent  on  each  page  and  per  question  were: 


Page 

Mean  (min) 

per  O  (min) 

1  Ionic 

3.33 

1.11 

New  SF  Plan 

3.67 

1.83 

Known  Plans 

6.00 

1.50 

New  Plan* 

10.75 

2.15 

Additional  Info 

20.75 

1.04 

Sum 

44.5 

•n  -  4 
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Numbers  of  Assists  and  Errors 

The  raw  numbers  of  assists  and  errors,  and  assists  and  errors  per  question  (corrected  for  missing 
data)  were: 


Page 

| 

< 

#  Errors 

Assists  per  Q 

Errors  per  Q 

1  lome 

5 

3 

1.57 

0.94 

New  SF  Plan 

7 

3 

3.5 

1.5 

Known  Plans 

9 

1 

2.25 

0.25 

New  Plan* 

9 

3 

1.21 

0.4 

Additional  Info 

II 

3 

0.55 

0.15 

•n  -  4 


There  were  no  rev  ersals  (backing  up  within  or  across  pages)  because  of  errors,  and  all  tasks  were 
completed  with  one  major  exception  (below).  Eight  errors  were  rated  as  being  a  “minor 
problem.”  These  are  described,  below.  No  error  was  rated  as  being  a  “show  stopper.” 

The  one  major  task  completion  failure  occurred  during  the  test  period  of  the  5,h  participant.  At 
approximately  08:45  CDT.  while  the  participant  was  using  the  Known  Plans  page,  the  F-PAS 
server  accessibility  slowed  dramatically,  with  pop-up  windows  requiring  several  minutes  to 
appear.  Accessibility  was  equally  slow  on  the  back-up  testing  system,  which  used  a  cellular 
modem  to  connect  to  the  Internet.  Since  two  different  Internet  access  methods  were  impeded,  it 
was  assumed  that  an  Internet  failure  had  occurred  somew  here  in  the  line  of  communication 
betw  een  San  Antonio  and  the  physical  location  of  the  F-PAS  serv  er,  or  that  there  had  been  a 
failure  at  the  server,  itself. 

The  5,h  participant  used  an  alternate  F-PAS  web  site  to  test  the  Additional  Information  page.  The 
server  accessibility  problem  continued  through  at  least  1 1 :00  CDT,  including  the  testing  period 
of  the  6lh  participant.  Thus,  that  participant  also  used  the  alternate  web  site  to  test  all  pages. 
Unfortunately,  the  alternate  web  site  had  no  access  to  the  core  software,  thus  these  two 
participants  were  not  able  to  test  the  New  Plan  page.  This  resulted  in  the  reduced  sample  size  (n 
4)  shown  above  for  the  mean  time,  assist  and  error  data. 

Our  subsequent  investigations  revealed  that  host  server  had  failed  and  that  the  Web  host  was 
somewhat  unreliable.  Consequently,  for  this  and  other  reasons  concerning  graphics-package 
compatibility,  a  change  was  made  subsequently  to  a  new  Web  host. 


Error  Descriptions 

The  eight  errors  rated  as  being  a  "minor  problem”  are  described  here.  First,  one  of  the 
participants  was  unable  to  detect  the  information  on  the  1  lome  page  indicating  that  the  software 
could  not  prevent  the  effect  of  fatigue  on  mental  performance. 


15 

Approved  for  public  release:  distribution  unlimited  Public  Attain  Case  tile  no.  09-295, 29  lune  21X19 


Second  and  third,  two  of  the  participants  were  unable  to  identify  two  of  the  four  "secrets"  to 
shiftwork  scheduling  that  were  listed  on  the  Home  page.  Fourth,  the  presence  of  the  calculation 
form  on  the  New  SF  Plan  page  interfered  with  a  participant's  ability  to  locate  the  Employment 
Ratio  calculator,  which  was  hyperl inked  from  that  page. 

Fifth,  one  participant  was  unable  to  determine  where  to  start  using  the  calculator  on  the  New  SF 
Plan  page,  and  (sixth)  was  unable  to  find  the  number  of  available,  qualified  people  who  would 
not  be  included  in  the  shift  schedule. 

Seventh,  on  the  Analysis  output  generated  from  the  New  Plan  page,  the  text  above  the  table  did 
not  allow  the  participant  to  identify  clearly  the  need  for  the  application  of  fatigue 
countermeasures  for  the  red-coded  shifts.  Eighth,  on  the  Additional  Information  page,  one 
participant  was  unable  to  determine  from  the  text  that  a  shiftwork  schedule  is  more  than  just  a 
shiftwork  plan  (rota). 

Errors  1,  2.  3.  and  8  appeared  to  be  generated  by  users  who  preferred  the  acquisition  of  data  from 
figures  and  charts  over  text.  Errors  4  through  7  appeared  to  be  generated  by  a  lack  of  adequate 
instructions  and/or  pointers  on  the  page. 


Information  Acquired  from  Assists 

The  assists  provided  by  the  observer  to  the  participants  revealed  other  weaknesses  in  the 
interface  design.  On  the  New  SF  Plan  page: 

•  The  concept  of  available,  qualified  people  who  would  not  be  included  in  the  shift 
schedule  was  not  well  explained  on  the  page. 

•  A  question  arose  about  the  fact  that  the  Panama  plan  was  available  and  generated  2 1 
extra  people  when  the  number  of  people  available  was  SI.  but  when  the  number  of 
available  people  was  cut  by  11.  the  Panama  plan  was  no  longer  available.  This  seemed 
counterintuitive. 

•  In  SF.  some  security  posts  are  not  manned  at  all  time.  I  low  can  this  variability  be 
reflected  in  the  interface?  Discussion  with  the  participant  suggested  two  approaches  to 
the  problem.  First,  the  Mission  Scheduler  Interface  could  be  linked  here  and  used  to  deal 
with  irregular  schedules  of  elements  or  sub-elements.  Second,  instead  of  dividing  an  SF 
squadron  administratively  into  (for  example)  four  flights  on  solely  the  basis  of  macro 
approaches  to  day  and  night  scheduling,  divide  the  squadron  into  elements  based  on 
shiftwork  demands.  For  example,  a  flight  or  element  that  deals  with  nuclear  weapon 
security  could  work  rapidly-rotating.  8-hour  shifts  with  no  variability  in  the  number  of 
posts  manned.  Meanwhile  a  flight  that  deals  with  access  control  inside  the  base  might  be 
split  into  three  elements  that  use  a  split-shift  schedule  for  reduced  staffing  at  night,  and  a 
flight  that  deals  with  gate  security  for  base  access  could  be  split  into  four  elements  that 
use  an  8 1 12  schedule  with  longer  shifts  on  weekends,  when  traffic  is  light.  Many 
variants  of  these  approaches  would  be  av  ailable  and  would  be  supported  by  the  New  SF 
Plan  page. 

•  In  SF.  not  all  personnel  are  qualified  for  all  duties.  How  can  this  variability  be  reflected 
in  the  interface?  Probably  with  the  same  approaches  as  above,  with  unqualified  people 
left  out  of  the  employment  ratio  calculation  for  the  more  demanding  duties. 
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•  The  employment  ratio  (calculated  by  a  link  from  the  New  SF  Plan  page)  may  also  be 
called  a  manning  factor. 

On  the  Known  Plans  page: 

•  The  abbreviations  "nil,"  “inefT  and  "Admin”  in  the  table  were  not  interpretable.  The 
table  needed  a  better  caption. 

•  The  “Complex”  column  did  not  belong  in  the  table.  It  needed  to  be  moved  to  a  well- 
labeled.  subsequent  paragraph. 

•  The  word  "interpolated”  in  the  5-crew  plan  pop-up  was  not  understandable. 

•  The  word  "solution”  in  the  pop-ups  was  contusing. 

On  the  New  Plan  page: 

•  The  need  to  click  either  the  “Cities"  or  the  “Bases”  button  to  allow  use  of  the  location 
drop-down  list  was  not  stated  on  the  page. 

•  The  error  message  (red  font)  was  difficult  to  detect. 

•  It  was  unclear  that  the  Analysis  was  being  produced  for  only  one  crew. 

•  For  the  Analysis  table,  there  was  no  instruction  to  hover  over  a  cell  to  see  the  average 
effectiveness  for  that  shift. 

•  On  the  right-hand  axis  of  the  Analysis  graph,  the  numbers  in  the  scale  for  BAC  had  no 
decimal  points. 

On  the  Additional  Information  page: 

•  The  relative  risk  graphs  in  pop-ups  were  confusing  to  one  participant,  especially  the 
fourth-night  effect  graph.  The  heights  of  the  bars  did  not  convey  a  message  that  is 
congruent  with  the  elevation  of  relative  risk.  For  example,  the  bar  that  indicated  a  36% 
increase  in  risk  was  four  times  as  tall  as  the  bar  that  indicated  a  relative  risk  of  1 .0. 

•  The  shift  system  pop-up  needed  a  clearer  discussion  of  the  relevance  and  non-relevance 
of  shiftwork  system  calculations  to  various  potential  users.  Also,  emphasis  was  needed 
for  the  fact  that  a  free  day  (F)  in  a  system  is  the  same  as  a  day  ofl'(O)  in  a  shiftwork  plan. 

•  The  importance  of  the  employment  ratio  calculation  was  not  explained  clearly  in  that 
pop-up. 

•  The  number  of  crews  graph,  in  that  pop-up.  needed  axis  labeling  congruent  with  the 
message  of  demands  made  upon  employee  and  employer. 

•  In  the  shift  start-time  pop-up.  the  word  “nadir”  was  difficult  to  interpret.  Also,  the  first 
sentence  had  a  typo. 

•  The  discussion  comparing  slow  and  fast  rotations,  in  the  rotation  pop-up.  was  not  as  clear 
as  it  needed  to  be  to  help  the  user  understand  that  we  know  very  little  about  which  one 
might  be  better. 


Discussion  Questions 

Specific  comments  made  by  the  participants  are  listed  here. 

What  were  your  objectives  os  you  tested  this  interface  /  after  using  the  first  couple  of  pages/? 
The  participants  provided  the  following  statements  in  response  to  this  question: 

•  Ease  of  use:  does  the  design  support  the  purpose? 
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•  Cari  the  software  help  me  design  ihe  best  possible  schedule  and  help  assure  that  1  have 
enough  people  to  man  it? 

•  How  may  I  apply  this  information?  Will  it  provide  support  material  for  briefings  to  the 
squadron  and  base  commanders? 

•  Can  1  navigate  with  ease?  Is  the  program  user-friendly? 

•  I  low  may  I  use  this  program? 

•  1  low  may  the  issue  of  varying  qualifications  of  personnel  be  addressed? 

Was  ihe  scope  of  I  he  usability  testing  that  you  Jill  adequate  to  meet  your  objectives? 

•  Yes.  but  actual  use  would  be  more  revealing  (n  2). 

•  Yes  (n  3). 

•  Better  to  use  a  scenario  adapted  from  an  actual  requirement,  such  as  the  security 
requirement  and  plan  at  Lackland  AFB.  Then,  how  to  move  from  12-hour  to  8-hour 
shirts?  How  to  introduce  a  5-crew  schedule? 

Could  the  Shiftwork  Scheduler  analysis  report  be  formatted  differently  to  better  assist  you  in 
scheduling?  (n  4) 

•  I  needed  this  when  1  was  scheduling  shiftwork. 

•  Add  the  hover  note  for  the  Analysis  table. 

Could  the  Shiftwork  Scheduler  analysis  graph  be  formatted  differently  to  belter  assist  you  in 
scheduling?  (n  4) 

•  The  length  of  the  [28-day]  graph  was  bothersome. 

•  The  graph  needs  labels  on  the  x-axis. 

•  Freeze  the  y-axis  labels  when  scrolling  left-right. 

•  The  graph  will  be  good  for  schedule-change  planning  and  for  briefings. 

What  other  improvements  should  be  made  to  the  Shiftwork  Scheduler  interface? 

•  The  overall  format  was  good. 

•  The  hyperlinks  were  good. 

•  The  hyperlinks  were  not  preferred,  but  no  alternate  approach  could  be  suggested. 

•  Add  Help. 

•  I  needed  this  when  1  was  scheduling  shiftwork  6  years  ago. 

•  Use  colors  to  help  draw  attention  to  objectives  of  relative  risk  and  number  of  crews 
graphs. 

•  Better  speed  [referring  specifically  to  the  loss  of  connectivity  described  above) 

•  Less  reading,  more  visuals. 

•  Tailor  to  varying  personnel  qualifications  [and.  by  extension,  to  variable  staffing  of 
posts]. 

In  response  to  these  questions  and  recommendations,  a  Help  file  was  constructed  and  the 
following  changes  were  made  to  the  interface: 

On  the  I  lome  Page: 

•  Text  added  indicating  that  the  software  could  not  prevent  the  effect  of  fatigue  on  mental 
performance. 
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On  the  New  SF  Plan  page: 

•  Added  a  calculator  icon 

•  Added  text  clarifying  the  concept  of  available,  qualified  people  who  would  not  be 
included  in  the  shift  schedule. 

•  The  question  about  the  Panama  plan  generating  2 1  extra  people  when  the  number  of 
people  available  was  81.  but  being  no  longer  available  when  the  number  of  av  ailable 
people  was  cut  by  1 1  was  resolved  as  a  non-problem.  It  was  due  to  the  correct 
application  of  the  employment  ratio  to  total  number  of  people. 

•  Added  text  that  discussed  possible  shiftwork  solutions  to  the  practice  of  not  stalling  some 
security  posts  at  some  times,  and  the  problem  of  not  all  personnel  being  qualified  for  all 
duties. 

•  Added  text  to  refer  to  the  employment  ratio  also  as  a  stalling  factor. 

In  the  Employment  Ratio  Calculator  pop-up: 

•  Added  an  error  check  to  protect  from  employment  ratio  <  I  errors. 

•  Changed  the  ratio  calculator  lines  from  fixed  words  to  defaulted  text  boxes. 

•  Added  the  calculator  icon  (above)  within  this  pop  up. 

On  the  Known  Plans  page: 

•  Removed  the  abbreviations  “nil  ”  “inefT  and  "Admin"  from  the  table. 

•  Moved  the  “Complex”  column  from  the  table  to  a  subsequent  paragraph. 

•  Replaced  the  word  "interpolated"  in  the  5-crew  plan  pop-up  with  “inserted  in." 

•  Defined  the  word  “solution." 

On  the  Additional  Information  page: 

•  Added  text  clarifying  that  a  shiftwork  schedule  is  more  than  just  a  shiftwork  plan  (rota). 

•  Replaced  the  relative  risk  graphs  with  tables  in  the  pop-ups. 

•  Added  text  in  the  shift  system  pop-up  to  clarify  the  relevance  and  non-relevance  of 
shilhvotk  system  calculations  to  various  potential  users,  and  to  help  show  that  that  a  free 
day  (F)  in  a  system  is  the  same  as  a  day  off(O)  in  a  shiftwork  plan. 

•  Added  text  to  help  show  the  importance  of  the  employment  ratio  calculation  in  that  pop¬ 
up. 

•  Provided  an  improved  number  of  crews  graph  in  that  pop-up.  changing  the  axis  labeling 
to  be  congruent  with  the  message  of  demands  made  upon  employee  and  employer. 

•  Modified  the  text  in  the  shift  start  time  pop-up. 

•  Modified  text  comparing  slow  and  fast  rotations  in  the  rotation  pop-up. 

The  following  tasks  were  postponed: 

•  Build  a  stand-alone  version  of  the  interface  (no  connection  to  SAFTE  engine  on  server). 
This  version  would  lack  the  New  Plan  page  and  would  be  hosted  on  a  SF  server. 

•  Put  BAC  and  relative  risk  truth  tables  in  examples. 
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DISCUSSION 


REQUIREMENTS  ANALYSIS 

This  discussion  focuses  on  several  of  the  nine  components  of  shiftwork  scheduling  and  refers  to 
several  of  the  nine  principles  of  shiftwork  scheduling  (Miller.  2006).  The  first  components  deal 
with  the  number  of  people  available  to  support  24/7  operations.  This  w  as.  undoubtedly,  the 
greatest  obstacle  faced  by  USAF  Security  Forces  at  the  time  of  this  requirements  analysis 
session.  The  next  components  deal  with  the  management  of  shiftworkers'  time.  Finally.  Security 
Forces  practices  are  addressed  with  respect  to  the  known  principles  of  shiftwork  scheduling. 


Number  of  People 

On  the  surface,  it  appeared  that  the  Security  Forces  were  attempting  to  use  four  crews  to  support 
24/7  operations.  This  would  be  a  good  thing.  In  regular,  non-combat  24/7  military  operations, 
the  number  of  crews  used  should  be  greater  than  the  number  of  shifts  per  day  so  that  at  least  one 
crew  is  off  each  day.  The  number  of  crews  sets  the  work  demand  in  a  regular  plan  (not  a 
schedule)-,  the  number  of  crews  defines  the  average  yearly,  weekly,  and  daily  amounts  of  time 
worked  by  an  individual.  Each  year  provides  (364  days  per  year  x  24  hours  per  day  ~)  8,736 
hours  to  be  worked  in  continuous  operations.  Each  crew  must  work  their  proportional  share  of 
the  year.  If  one  specifies  four  crews,  then  eaeli  person  in  the  crew  must  work  2.184  hours  per 
year,  an  average  of  42  hours  per  week.  Thus,  the  4-crew  solution  approximates  the  usual  work 
demand  placed  on  weekday-only  workers. 

The  graph  of  the  relationship  between  the  average  amount  of  time  an  indiv  idual  works  and  the 
number  of  crews  used  is  not  a  straight  line  (Miller.  2006).  Because  the  line  is  curved,  the  4-crew 
solution  provides  the  optimal  balance  between  (1)  the  work,  health,  social,  and  safety  demands 
placed  upon  the  shiftworker  in  terms  of  hours  worked  per  unit  time,  and  (2)  the  personnel  cost  to 
the  employer  for  safe  and  productive  system  operation  in  terms  of  the  number  of  crews  needed. 

The  Security  Forces  were  using  the  4-crew  solution  inappropriately  because  of  their  general 
personnel  shortage.  Irregular  schedules  were  being  used  to  increase  the  ratio  of  work  days  to 
days  off.  For  example,  if  a  ratio  of  6  days  on  to  3  days  off  was  being  used  and  the  security 
demand  increased,  the  ratio  might  be  changed  to  6  days  on  and  2  days  off  to  meet  the  new 
demand.  Of  course,  the  most  appropriate  way  to  meet  the  higher  demand  would  be  to  have 
Security  Forces  personnel  in  reserve  who  could  be  used.  Unfortunately,  irregular  schedules 
often  generate  morale  problems  because  they  violate  the  shiftwork  scheduling  principles  of 
predictability,  equity  and  good-quality  time  off  (Miller.  2006). 

In  addition  to  having  enough  people  for  four  crews,  the  employment  ratio  (ratio  of  work  to  free 
days)  must  be  great  enough  to  deal  with  holidays,  sick  leave,  annual  leave  and  other 
administrative  and  training  time.  The  total  work  force  needed  is  the  product  of  the  minimum 
number  of  people  for  a  single  crew,  the  number  of  crew  s  required,  and  the  employment  ratio. 

There  are  many  cases  in  the  military  in  which  training  is  so  demanding  of  shiftworkers’  time  that 
a  4-crew  solution  seems  impractical.  A  solution  to  this  problem  is  to  use  five  crews.  Typically, 
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a  5-crew  solution  is  created  by  adding  a  51’1  crew  to  a  4-crew  solution  and  placing  one  crew  in  a 
9-to-5  (or  other)  schedule  for  training  for  a  continuous  period  (Miller,  2006).  This  option  was 
discussed  with  the  SMEs. 


Time  Management 

There  are  two  solutions  that  may  be  applied  to  the  24/7  scheduling  problem:  fixed  shifts  and 
rotating  shifts.  With  respect  to  rotating  shifts,  there  are  two  options:  fast  and  slow  rotation.  On 
fixed  shifts,  the  worker  always  works  the  same  shift:  for  example,  permanent  days  or  permanent 
nights.  Most  fixed-shift  workers  tend  to  become  dissatisfied  with  fixed  shifts  and  wish  to 
“graduate"  to  day  shifts.  Besides  obvious  inequities  for  workers  between  fixed  day  and  night 
shifts,  fixed  night-shift  workers  usually  do  not  keep  their  day-sleep,  night-work  schedule  on  days 
otr.  Thus,  fixed  shift  schedules  acquire  much  of  the  nature  of  poorly-scheduled  rotating  shifts. 

Many  24/7  operations  are  supported  by  rotating  shiftwork,  in  which  the  worker  changes  (rotates, 
as  the  hands  of  a  clock  rotate  around  the  clock  face)  from  one  shift  to  another  at  some  specified 
interval,  be  it  “fast"  or  “slow."  There  are  infinite  numbers  of  possible  rotating  shiftwork  plans. 
The  principle-based  scheduling  approach  planned  for  the  F-PAS  interface  will  constrain  the 
infinite  number  of  possible  rotating  shiftwork  plans  to  those  that  are  practical  to  implement  and 
least  harmful  to  shiftworker  health,  job  performance  and  altitude. 

In  theory,  shift  length  may  be  any  amount  of  time  up  through  24  hours.  In  practice,  only  the 
even  factors  of  24  hours  (2.  4.  6.  8,  12  or  24  hours)  are  useful.  The  most-debated  issue 
concerning  shift  length  these  days  concerns  the  use  of  12-h  shifts  instead  of  8-h  shifts.  In  regular 
schedules,  shiftworkers  are  quick  to  notice  that,  due  to  the  work  compression  afforded  by  12-h 
shifts,  they  can  have  longer  continuous  periods  off  when  working  12-h  shifts  than  when  working 
8-h  shifts.  These  longer  time-off  periods  are  viewed  as  being  of  high  value,  and  so  shiftw  orkers 
often  prefer  12-h  shifts  to  8-h  shifts.  However,  there  is  well-placed  concern  about  the  use  of  the 
12-h  shift  length  when  safety  issues  are  present.  For  example,  combined  data  from  three 
excellent  field  studies  demonstrated  that  the  “...risk  [of  injuries  and  accidents]  increased  in  an 
approximately  exponential  fashion  with  time  on  shift  such  that  in  the  twelfth  hour  it  was  more 
than  double  that  during  the  first  8  h"  (Folkard  and  Tucker.  2003).  If  one  were  to  investigate,  for 
example,  accidental  firearm  discharges  within  the  work  shift  for  Security  Forces  personnel, 
controlling  for  time  of  day.  it  is  likely  that  the  9“  through  I2,h  hours  of  the  work  shift  would  be 
overrepresented  compared  to  the  1*  through  8lh  hours. 

Shift  overlap  (“hand  off')  is  the  time  spent  before  and  after  a  shift  to  transition  control  of  the 
system  from  the  crew  ending  its  shift  to  the  crew  starting  its  shift.  Overlap  is  not  used  in 
calculations  and  comparisons  of  shiftwork  plans  because  overlap  requirements  vary  only  as  a 
function  of  the  work  domain.  However,  overlap  must  be  used  for  calculations  of  total  numbers 
of  hours  worked,  after  a  shiftwork  plan  has  been  selected.  Often,  excessive  overlap  times  are 
viewed  as  necessary  by  management.  Unfortunately,  these  long  overlaps  extend  the  workday  so 
much  that  there  is  a  high  risk  of  cumulative  fatigue,  followed  by  chronic  fatigue  and 
accompanying  morale  and  retention  problems. 
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Security  Forces  Practices 


As  with  many  military  organizations,  USAF  Security  Forces  shift  work  practices  are  limited  by 
insufficient  numbers  of  SF  personnel  at  most  locations.  When  a  unit  has  a  sufficient  number  of 
people  to  stall  a  standard.  4-crew  system,  planners  tend  to  use  the  "Panama"  or  “every-other- 
vveekend-off"  rota  in  a  shiftwork  system  that  uses  12-hour  shifts  and  7  work  days  combined  with 
7  free  days  per  cycle  (7W:7F  system;  the  Panama  plan  is  described  fully  in  Miller.  2006).  The 
rotation  between  day  and  night  shifts  usually  occurs  after  two  cycles,  i.e.,  once  every  28  days  (a 
“slow"  rotation).  Working  days  or  nights,  an  individual  follows  this  work-rest  sequence  for  each 
14-day  cycle:  2  days  on.  2  days  off.  3  days  on.  2  days  off.  2  days  on.  and  3  days  off.  The  series 
of  3  days  on  and  the  series  of  3  days  off  both  fall  on  Friday,  Saturday  and  Sunday,  giving  an 
individual  every  other  weekend  off.  However,  the  Fridays  “off’  during  a  night's  night  shift 
month  should  be  viewed  as  recovery  days.  Thus,  across  364  days  (twenty-six  14-day  cycles),  an 
individual  has  26  w  eekends,  or  52  weekend  days,  off  per  year.  That  is  only  half  of  the  number 
recommended  by  the  shiftwork  principles  (Miller,  2006).  Thus,  although  the  Panama  plan  is  a 
relatively  good  12-hour  shiftwork  plan  (rota),  it  is  weak  in  this  regard  and  management  should 
expect  shiftworkers  to  perceive  inequity  with  weekday-only  workers  who  have  52  weekends  off 
per  year.  Table  HI  shows  the  shift  alignment  for  the  14-day  cycle  of  the  Panama  plan.  Although 
the  equity  of  this  plan  could  be  improved  by  converting  it  to  a  5-crew  plan  by  placing  one 
additional  crew  on  a  weekday-only  schedule  during  each  cycle,  the  lack  of  a  sufficient  number  of 
SF  personnel  probably  make  this  impossible  at  present.  In  fact.  USAF  Security  Forces  rarely 
have  the  luxury  of  using  the  Panama  plan. 


Table  111.  Shift  alignment  table  for  the  14-day  cycle  of  the  Panama  Plan 


Day 

Week 

Mon 

Tue 

Wed 

Thu 

Fri 

Sat 

Sun 

1 

W 

W 

R 

O 

W 

W 

W 

2 

R 

O 

W 

W 

R 

<) 

o 

Note:  W  is  a  workday  <  1 
day  off;  R  is  a  recovery  < 

12-h  day  or  night  shift  with  rotation  once  every  28  days);  O  is  a 
lay  if  W  is  the  night  shirt,  else  it  is  a  day  off(O). 

More  typically,  because  insufficient  numbers  of  SF  personnel  are  present  at  a  location,  a  legacy 
6\V:3F,  4-crew  system  w  ith  12-hour  shifts  is  used  as  a  default  (DDDDDDO(K)  or 
NNNNNNROO,  where  D  is  a  day  shift.  N  is  a  night  shift.  R  is  a  recovery  day.  and  O  is  a  day 
off).  The  reason  that  this  is  a  “legacy”  system  is  that  the  6W:3F  system  is  best  suited  for  8-hour 
shifts,  but  SF  adopted  12-hour  shifts  to  deal  with  its  personnel  shortage.  The  result  of  this 
change  is  that,  instead  of  having  half  of  the  work  force  covering  two  shifts  per  day.  as  is  normal 
for  a  system  that  uses  4-crews  and  1 2-hour  shifts.  SF  uses  /  of  the  workforce  during  each  24- 
hour  period  (as  one  would  expect  for  8-hour  shifts  and  4  crews).  This  practice  increased  the 
individual's  work  demand  from  the  standard  4-crew  average  workweek  of  42  hours  to  an  average 
of  56  hours  per  week,  the  same  average  work  demand  experienced  by  workers  in  3-crew 
systems.  Reviewing  the  shiftwork  principles  and  supporting  industrial  mishap  data,  one  notes 
that  six  nights  of  work  in  a  row  is  too  much.  A  rapidly  rotating  DDDNNNROO  plan  with  12- 
hour  shifts  (still  in  the  6W:3F  system)  might  be  more  appropriate  in  many  cases. 
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Security  Force  units  vary  the  ratio  of  work  to  free  days  away  from  6W:3F  within  elements  and 
flights  as  the  work  demand  changes.  The  numbers  of  posts  that  are  manned  each  day  also  vary 
as  a  function  of  work  demand,  with  low-priority  posts  being  unmanned  during  periods  of  high 
work  demand.  There  are  several  problems  associated  with  the  varying  of  the  W:F  ratio.  The 
main  problem  is  that  the  varying  of  the  ratio  to  meet  transiently  increased  security  demands  can 
lead  to  ml  hoc  scheduling.  As  mentioned,  above,  the  resulting  irregular  schedules  can  generate 
morale  problems  because  they  violate  the  shiftwork  scheduling  principles  of  predictability, 
equity  and  good-quality  time  off. 


Interface  Overview 

Our  general  plan  for  the  F-PAS  interfaces  was  to  use  two  main  conceptual  pathways  for 
scheduling:  regular  schedules  and  irregular  schedules.  Regular  schedules  for  shift  workers 
would  be  supported  by  the  sequential,  prescriptive  approach  documented  in  Miller  (2006).  This 
approach  would  generate  schedules  based  on  a  set  of  algorithms  using  inputs  such  as  the  number 
of  crews,  shift  length,  shift  plan,  start  date,  and  start  time.  Schedules  such  as  the  Panama  plan 
would  result  from  this  prescriptive  approach.  Output  would  consist  of  a  listing  of  24-hour 
periods  showing  the  performance  effectiveness  for  a  crew  across  the  24-hour  period  using  color 
(green,  yellow,  red),  average  performance  effectiveness  values,  and  the  time  and  amount  of  the 
lowest  value  for  a  shift.  Tabular  output  would  also  be  available  for  import  into  a  spreadsheet. 

When  too  few  personnel  were  available  to  meet  the  work  demand  with  a  regular  schedule,  a 
scheduler  would  use  an  irregular  schedule  to  accomplish  the  required  work.  Irregular  schedules 
would  be  created  and  evaluated  using  procedures  similar  to  existing  FAST1'1  input  methods. 

This  approach  was  descriptive  rather  than  prescriptive;  the  user  generates  the  schedule  without 
any  constraints  and  the  model  is  used  to  evaluate  it.  Output  would  be  similar  to  that  for  Regular 
schedules.  These  needs  were  to  be  supported  by  the  Mission  Scheduler  Interface  (Eddy  et  al., 
2008  (. 

WALK-THROUGH 

The  three  walks  through  the  developing  SSI  were  informative  with  respect  to  interface  design. 
Because  there  was  general  ignorance  across  the  Air  force  with  respect  to  the  design  of  regular, 
rotating  shiftwork  schedules  (AF1A.  2004),  we  took  a  best  guess  at  how  to  design  the  interface, 
basing  our  guess  on  the  results  of  the  requirements  analysis  and  the  information  provided  in 
Miller's  scheduling  manual  (2006).  The  approach  was  prescriptive,  pushing  the  scheduler 
toward  a  4-crew  solution  with  either  8-  or  12-hour  shift  lengths.  The  approach  was  also 
educational,  providing  background  information  in  pop-up  windows  and  on  the  Additional 
Information  page. 

The  first  cut  was  well  received.  The  major  change  request  was  to  break  the  single-page  interface 
into  several  pages.  One  interesting  innovation  that  was  suggested  was  the  tabular  display  of 
cognitive  effectiveness  for  one  crew  as  a  function  of  shirts  and  days.  Another  was  the  selection 
of  the  color  blue  to  represent  impairment  during  the  pre-dawn  hours  of  the  night  shift.  These  and 
any  other  changes  were  made  to  the  interface  during  its  development. 
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INSPECTION  EVALUATION 


Generally,  the  overall  ratings  of  the  interface  for  ease  of  application  performance  arul  overall 
function  were  “acceptable"  and  “very  acceptable"  ( 1  s  and  2s  on  a  scale  of  1  to  5.  where  1  was 
best  and  5  was  worst).  The  mean  amount  of  time  spent  on  each  scenario  question  ranged  from 
1 .04  min  on  the  Additional  Information  page  to  2. 1 5  min  on  the  New  Plan  page.  The  mean  times 
appeared  to  agree  with  the  difficulty  levels  of  the  pages.  The  New  Plan  and  New  SF  Plan  pages 
were  less  informational  in  design  and  probably  required  longer  decision  times  than  the  other 
pages. 

The  mean  numbers  of  assists  per  scenario  question  ranged  from  0.55  to  3.5,  with  the  highest 
numbers  for  the  New  SF  Plan  and  Known  Plans  pages,  which  were  related  by  references  to 
known  shiftwork  plans.  The  mean  numbers  of  errors  per  scenario  question  ranged  from  0.15  to 
1 .5.  with  the  Home  and  New  SF  Plan  pages  recording  the  highest  number.  Generally,  the  New 
SF  Plan  page  seemed  to  generate  the  most  usability  problems  among  the  five  pages.  The 
descriptions  of  eight  specific  errors,  our  “assists"  and  the  answers  to  our  other  usability  questions 
allowed  us  to  design  improvements  that  were  needed  on  the  various  pages.  We  created  a  1  lelp 
file  and  made  these  changes,  about  20  in  number,  just  prior  to  the  end  of  the  technical  portion  of 
the  contract. 
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CONCLUSIONS 


The  personnel  shortage  faced  by  USAF  Security  Forces  may  be  representative  of  many  military 
and  civilian  groups  who  must  support  24/7  operations.  Faced  with  the  lack  of  adequate  numbers 
of  skilled  personnel  to  support  the  desired  4-crew  solution,  along  with  an  adequate  staffing  ratio, 
organizations  are  unable  to  establish  acceptable  shiftwork  schedules.  1  lowever.  part  of  the 
problem  that  SF  faced  was  the  absence  of  policies,  procedures  and  tools  that  could  provide 
assistance  for  the  development  of  any  reasonably  good  shiftw  ork  schedule.  The  requirements 
assessment  revealed  the  impact  of  that  absence  w  ithin  SF. 

If  adequate  policies,  procedures  and  tools  had  been  available.  SF  might  have  realized  that  the 
legacy  12-hour,  slowly-rotating.  6W:3F  system  that  they  adopted  was  not  feasible  because  it 
caused  too  many  sequential  nights  of  work  for  individuals  on  the  night  shift.  They  might  have 
adopted,  instead,  the  superior  rapidly-rotating  plan  suggested,  above. 

Our  iterative  sessions  with  SMEs  indicated  that  the  shiftwork  scheduler's  interface  should 
provide  a  mechanism  that  allows  schedulers  to  make  schedules  that  meet  the  demands  of  the 
work,  informed  decisions  with  respect  to  all  of  the  principles  and  components  of  shiftwork 
scheduling,  and  "what-if"  comparisons  across  various  alternative  schedules.  The  comparisons 
should  be  made  in  terms  of  a  specific  schedule's  compliance  with  the  shiftwork  scheduling 
principles  and  in  terms  of  points  within  a  schedule  that  are  particularly  fatiguing,  identified  by 
the  SAFTE  model.  In  the  latter  case,  for  example,  the  model  should  identify  the  likelihood  that 
an  individual  beginning  the  first  day  shift  of  a  cycle  has  not  fully  recovered  from  previous  night 
work  and  is,  therefore,  at  an  elevated  risk  of  a  mishap.  The  interface  should  suggest  the  use  of 
countermeasures  when  mishap  risk  is  higher  than  normal. 

The  input  component  of  the  interface  should  constrain  the  potentially  infinite  possibilities  for 
shiftwork  schedules  to  those  that  make  the  most  sense  in  terms  of  the  nine  shiftw  ork  principles 
(Miller.  2006).  The  input  component  should  be  both  minimal  in  scope  and  provide  extensive 
optional  guidance  with  respect  to  the  nine  principles  and  nine  components  of  shiftw  ork 
scheduling.  The  shiftwork  scheduling  manual  (Miller,  2006)  should  be  available  for  study  and 
review.  The  output  component  of  the  interface  should  identify  clearly  the  "fatigue  points"  in  the 
proposed  schedule,  based  upon  SAFTE™  calculations,  and  the  compliance  of  the  schedule  with 
the  nine  principles.  The  output  component  should  also  allow  side-by-side  comparisons  of 
several  candidate  schedules.  The  walk-through  and  inspection  evaluation  processes  indicated 
that  most  of  these  requirements  were  met  reasonably  well  and  that  potential  users  were  able  to 
operate  the  interface  easily. 
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APPENDIX  A:  F-PAS  SHIFTWORK  SCHEDULER  USABILITY  QUESTIONNAIRE 

NTL  Inc..  April  2008 

Years  of  active  duty: 

Years  spent  in  shiftwork  and' or  night  work  as  worker  and/or  scheduler:  _ 

FAST  user?  Yes  No  (circle  one) 

Overall  Ratings 

Please  rale  the  ease  of  application  of  the  Sltifhvotk  Scheduler  interface  to  the  intended  task:  the 
simplicity  with  which  the  Shiftwork  Scheduler  interface  can  be  employed  to  help  with  scheduling.  In  an 
ideal  world,  the  interlace  would  be  totally  natural  and  predictable  in  behavior.  Nothing  should  obstruct 
your  progress  in  completing  this  task. 

Very  acceptable  ( 1 )  (circle  one) 

Acceptable  (2) 

Borderline  (3) 

Unacceptable  (4) 

Very  unacceptable  (5) 

Rate  the  performance  of  the  Shiftw  ork  Scheduler  interface:  the  speed  with  which  the  interlace  responds 
to  requests. 

•  Very  acceptable  ( 1 )  (circle  one) 

•  Acceptable  (2) 

•  Borderline  (3) 

•  Unacceptable  (4) 

•  Very  unacceptable  (5) 

Rate  the  support  information  for  the  Shiftw  ork  Scheduler  interface:  the  information  available  to  acquire, 
use  and  support  the  Shiftwork  Scheduler  interlace,  kneompasses  initial  instructions,  user  guides, 
tutorials,  integrated  assistance. 

•  Very  acceptable  ( 1 )  (circle  one) 

•  Acceptable  (2) 

•  Borderline  (3) 

•  Unacceptable  (4) 

•  Very  unacceptable  (5) 


Rate  the  Shiftwork  Scheduler  interface's  function:  the  overall  capabilities  of  the  Shiftwork  Scheduler 
interface. 

•  Very  acceptable  ( 1 )  (circle  one) 

•  Acceptable  (2) 

•  Borderline  (3) 

•  Unacceptable  (4) 

•  Very  unacceptable  (5) 


Please  discuss  with  the  observer: 

•  What  were  your  objectives  as  you  tested  this  interface? 

•  Was  the  scope  of  the  usability  testing  that  you  did  adequate  to  meet  your  objectives? 

•  Could  the  Shiftwork  Scheduler  analysis  report  be  formatted  differently  to  better  assist  you  in 
scheduling? 

•  Could  the  Shiftwork  Scheduler  analysis  graph  be  formatted  dilYcrently  to  better  assist  you  in 
scheduling? 

•  What  other  improvements  should  be  made  to  the  Shiftwork  Scheduler  interface? 
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APPENDIX  B:  NTI  F-PAS  SHIFTWORK  SCHEDULER  TOOL  USABILITY  DATA 


Data: 

On  a  separate  page,  keep  orderly,  transeribable  notes  of  the  pathways  the  participants  take,  problems  participants  have  and  what  participants  say  as 
they  work.  Definitions  for  the  table,  below: 

•  Number  of  sttbla.sk  assists:  When  the  participant  cannot  proceed  on  a  subtask .  the  observer  gives  direct  procedural  help  to  allow  the  test  to 
proceed. 

•  Number  of  subtask  errors:  Instances  where  test  participant  had  to  attempt  portions  of  the  task  more  than  once. 

•  Number  of  sub  task  reversals:  Number  of  tunes  participant  had  to  ''back  up  to  find  something  on  a  previous  page  that  they  needed  on  the 
current  page. 

•  Subtask  completion  { Y/N):  Yes  =  complete  and  correct  achievement  of  subtask  goal. 

•  Problem  severity  ( 0/1/2 I:  0  no  problem;  1  minor  (users  are  annoyed,  but  this  does  not  keep  them  from  completing  the  scenario);  2  show 
stopper  (if  we  don't  fix  this,  users  will  not  be  able  to  complete  the  scenario;  and’or  many  users  will  be  frustrated,  and  they  may  give  up). 


Subtask 

#  Assists 

#  Errors 

#  Reversals 

Severity 

Completion 

1 .  Start  time  at  Home  page: 

NA 

NA 

NA 

NA 

NA 

2.  Q1  •  Will  the  Shiftwork  Scheduler  generate  a  24/7  plan  that  prevents  the  effects  of  fatigue 
on  mental  performance?  (No) 

0  1  2 

Yes  No 

3.  Q2-  The  four  main  secrets  to  shiftwork  scheduling  include:  (Worker  satisfaction.  Having 
enough  people) 

0  1  2 

Yes  No 

4.  03-  When  should  fatigue  countermeasures  be  used?  (At  predicted  fatigue  points) 

0  1  2 

Yes  No 

5.  End  time: 

NA 

NA 

NA 

NA 

NA 

6.  Start  time  at  New  SF  Plan  page: 

NA 

NA 

NA 

NA 

NA 

7.  Q4.  Use  the  Employment  Ratio  Calculator.  If  there  are  10  holidays,  14  vacation  days,  14 
sick  days.  24  training  days,  and  60  deployment  days  allotted  per  year,  15  people  arc 
needed  per  crew,  and  4  crews  are  needed,  a.  What  is  the  employment  ratio?  ( 1.34);  b. 

How  many  people  do  you  need?  (81) 

0  1  2 

Yes  No 

8.  Q5.  Close  the  Employment  Ratio  Calculator.  You  have  81  people  available.  15  posts  to 
be  manned  and  an  employment  ratio  of  1 .34.  For  the  Panama  plan,  how  many  people  will 
not  be  on  the  shiftwork  schedule  at  any  given  time?  (21 )  If  your  manning  is  cut  10%  to  73 
people,  can  you  still  use  the  Panama  plan?  (No) 

0  1  2 

Yes  No 

9.  End  time: 

NA 

NA 

NA 

NA 

NA 

10.  Start  time  at  the  Known  Plans  page: 

NA 

NA 

NA 

NA 

NA 
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Subtask 

«  Assists 

#  Errors 

*  Reversals 

Severity 

Completion 

1 1 .  Q6.  If  you  wish  to  implement  the  DDNNOOOO  plan,  a.  What  is  the  shift  length?  ( 12 
hours)  b.  How  many  crews  do  you  need?  (4) 

0  1  2 

Yes  No 

1 2.  07*  If  you  wish  to  use  a  5-crcw  plan  so  that  one  crew  is  always  in  training  or 

accomplishing  administrative  duties,  what  kind  of  a  plan  would  you  use  as  a  basis  for 
building  the  5-crew  plan?  (4  Crew  Plan) 

ffl 

Yes  No 

1 3.  Q8.  If  you  have  a  complex  schedule  with  a  low  work  demand  at  night  and,  thus,  need  only 
half  as  many  people  working  at  night,  what  kind  of  plan  might  meet  your  needs?  (  Split 
Crew) 

0  1  2 

Yes  No 

14.  Q9.  If  you  have  a  complex  schedule  with  a  low  work  demand  on  weekends  and.  thus,  may 
use  a  longer  shift  length  on  weekends,  what  kind  of  plan  might  meet  your  needs?  (8*12 
Plan) 

0  1  2 

Yes  No 

15.  kndtime: 

NA 

NA 

NA 

NA 

NA 

16.  Start  time  at  New  Plan  page: 

NA 

NA 

NA 

NA 

NA 

1 7.  Q10.  Refer  to  the  Known  Plans  page  to  get  the  shift  sequence  from  the  Metropolitan  rota. 
Sequence:  (DDSSNNOO)  Enter  this  sequence  on  the  New  Plans  page.  Use  1  April  2008 
as  a  start  date,  07:00  as  a  start  time.  Kelly  APB  as  the  location,  and  no  shift  overlap.  Click 
on  the  "Analysis**  button,  a.  What  is  the  lowest  mental  performance  for  the  firet  night 
shift?  (71%)  h.  What  is  the  average  mental  performance  for  the  first  night  shift?  (76%) 
(hover  the  cursor  over  the  cell)  c.  Should  fatigue  countermeasures  be  applied  dunng  this 
shift  for  safety  sensitive  jobs?  (Yes)  d.  What  is  the  basis  for  a  job  being  labeled  as 
"safety-sensitive?**  (risk  to  the  public) 

0  1  2 

Yes  No 

IS.  Ql  1.  Close  the  Analysis  page  and  refer  again  to  the  New  Plan  page.  Change  the  shift 
length  to  12  hours  (don’t  change  anything  else).  Click  on  the  "Analysis**  button.  What 
happens?  (Error:  12-hr  plans  don’t  have  swing  shifts) 

0  1  2 

Yes  No 

19.  Q12.  Enter  one  28-day  cycle  of  the  Panama  plan  sequence  from  the  Known  Plans  page  on 
the  New  Plan  page.  That  full  cycle  is  DDOODDDCX)DLXX)ONNOONNNOONNOOO. 
Use  1  April  2008  as  a  start  date.  07:00  as  a  start  time.  Kelly  AEB  as  the  location,  and  a  1- 
hour  shift  overlap.  Click  on  the  "Analysis**  button,  a.  What  is  the  lowest  mental 
performance  for  the  second  day  shift?  (78%)  b.  What  is  the  average  mental  performance 
for  the  second  day  shift?  (81%)  (hover  the  cursor  over  the  cell)  c.  What  is  the  lowest 

BAC  equivalent  reached  on  the  last  night  shift?  (0.008) 

0  1  2 

Yes  No 

20.  Q13.  Close  the  Analysis  page  and  refer  again  to  the  New  Plan  page.  The  Analysis  page 
that  was  displayed  was  for  how  many  crews?  ( 1 ) 

0  1  2 

Yes  No 
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Subtask 

«  Assists 

#  Furors 

#  Reversals 

Severity 

Completion 

21.  Q14.  Docs  the  Shiftwork  Scheduler  show  you  how  many  days  should  elapse  between  one 
crew  starling  the  cycle  and  another  crew  starting  the  cycle?  (No) 

0  1  2 

Yes  No 

22.  tend  time: 

NA 

NA 

NA 

NA 

NA 

23.  Start  time  at  Additional  Information  page: 

NA 

NA 

NA 

NA 

NA 

24.  015.  During  what  part  of  the  normal  circadian  rhvthm  are  speed  and  accuracy  above 
average?  (07:00  to  19:00) 

0  1  2 

Yes  No 

25.  Q16.  What  percentage  of  accident  risk  elevation  occurs  during  the  night  shift  compared  to 
the  day  shift?  (30%) 

0  1  2 

Yes  No 

26.  017-  What  percentage  of  accident  risk  elevation  occur*  on  the  fourth  night  shift  compared 
to  the  first  night  shift?  (36%) 

0  1  2 

Yes  No 

27.  Q1X.  When  cognitive  (mental)  effectiveness  is  78%,  what  is  the  equivalent  a.  BAC? 
(0.05%)  b.  Industrial  accident  risk?  <  1 .28) 

0  1  2 

Yes  No 

2S.  019.  In  one  word,  what  is  shiftwork  scheduling  principle  number  7?  (Kquity) 

0  1  2 

Yes  No 

29.020.  In  shiftwork  system  analysis,  what  docs  a.  W  mean?  (Work  period)  b.  F  mean? 
(Free  period) 

0  1  2 

Yes  No 

30.  Q2I-  W hat  is  a  useful  shiftwork  svstem  for  four  crews  and  12-hour  shifts?  (2n\V:2nF) 

0  1  2 

Yes  No 

3 1 .  Q22.  In  the  notation  for  a  shiftwork  plan  (rota),  a.  D  means  (DAY)  shift  b.  S  means 
(SWING)  shift  c.  N  means  (NIGHT)  shift  d.  O  means  (OFF)  shift 

0  1  2 

Yes  No 

32.  023.  A  shiftwork  schedule  is  simplv  a  shiftwork  plan  (rota).  (False) 

0  1  2 

Yes  No 

33.  024.  The  relationship  between  the  average  number  of  hours  worked  by  a  crew  and  the 
number  of  crews  is  a  straight  line.  (False) 

0  1  2 

Yes  No 

34.  025.  The  optimal  number  of  crews  to  use  in  terms  of  demands  on  both  the  employer  and 
employee  is  (4) 

0  1  2 

Yes  No 

35.  026.  In  two  words,  the  importance  of  calculating  the  employment  ratio  accurately  is  to 
assure  what?  (Numher  of  people  needed) 

0  1  2 

Yes  No 

36.  027.  A  slow  rotation  is  superior  to  a  fast  rotation  in  terms  of  safely  and  productivity. 

(False) 

0  1  2 

Yes  No 

37.  Q2X.  A  “forward”  rotation  means  that  a  crew  rotates  to  a  shift  that  starts  (later)  on  the 
clock. 

0  1  2 

Yes  No 
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Subtask 

«  Assists 

a  Errors 

*  Reversals 

Severity 

Completion 

38.  029.  Conccmina  shift  length.  llic  risk  of  an  accidcnl  in  the  1 2th  hour  of  a  shift  is  (2.2) 
times  greater  than  during  the  first  eight  hours  of  the  shift. 

0  1  2 

Yes  No 

39.  030.  Shift  overlap  is  not  used  in  calculations  and  comparisons  of  shiftwork  plans.  (True) 

0  1  2 

Yes  No 

40.  031.  Shift  differentials  mav  be  provided  bv  adjusting  the  numbers  of  hours  worked  on 
dift'erent  shifts.  (True) 

0  1  2 

Yes  No 

41 .  032.  What  scheduling  principle  is  supported  when  the  shiftwork  plan  is  aligned  to  provide 
the  highest  possible  number  of  weekend  days  off?  (Number  5) 

0  1  2 

Yes  No 

42.  033.  If  the  shift  start  time  is  selected  carefully,  then  both  (day)  and  (swing)  shift  workers 
will  have  the  opportunity  to  get  a  good  night  of  sleep. 

0  1  2 

Yes  No 

43.  Q34.  Night  workers  arc  often  at  risk  for  (driving)  accidents  after  work. 

0  1  2 

Yes  No 

44.  I£nd  time: 

NA 

NA 

NA 

NA 

NA 

Comments: 


31 

Approved  iii  public  release;  di'lnbution  unlimited.  Public  Allairs  Case  file  no.  09-295.  29  June  21X19 


